MIDRC-MetricTree: a decision tree-based tool for recommending performance metrics in artificial intelligence-assisted medical image analysis.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) was created to facilitate medical imaging machine learning (ML) research for tasks including early detection, diagnosis, prognosis, and assessment of treatment response related to the coronavirus disease 2019 pandemic and beyond. The purpose of this work was to create a publicly available metrology resource to assist researchers in evaluating the performance of their medical image analysis ML algorithms. Approach: An interactive decision tree, called MIDRC-MetricTree, has been developed, organized by the type of task that the ML algorithm was trained to perform. The criteria for this decision tree were that (1) users can select information such as the type of task, the nature of the reference standard, and the type of the algorithm output and (2) based on the user input, recommendations are provided regarding appropriate performance evaluation approaches and metrics, including literature references and, when possible, links to publicly available software/code as well as short tutorial videos. Results: Five types of tasks were identified for the decision tree: (a) classification, (b) detection/localization, (c) segmentation, (d) time-to-event (TTE) analysis, and (e) estimation. As an example, the classification branch of the decision tree includes two-class (binary) and multiclass classification tasks and provides suggestions for methods, metrics, software/code recommendations, and literature references for situations where the algorithm produces either binary or non-binary (e.g., continuous) output and for reference standards with negligible or non-negligible variability and unreliability. Conclusions: The publicly available decision tree is a resource to assist researchers in conducting task-specific performance evaluations, including classification, detection/localization, segmentation, TTE, and estimation tasks.

A Guideline for Open-Source Tools to Make Medical Imaging Data Ready for Artificial Intelligence Applications: A Society of Imaging Informatics in Medicine (SIIM) Survey.

In recent years, the role of Artificial Intelligence (AI) in medical imaging has become increasingly prominent, with the majority of AI applications approved by the FDA being in imaging and radiology in 2023. The surge in AI model development to tackle clinical challenges underscores the necessity for preparing high-quality medical imaging data. Proper data preparation is crucial as it fosters the creation of standardized and reproducible AI models while minimizing biases. Data curation transforms raw data into a valuable, organized, and dependable resource and is a fundamental process to the success of machine learning and analytical projects. Considering the plethora of available tools for data curation in different stages, it is crucial to stay informed about the most relevant tools within specific research areas. In the current work, we propose a descriptive outline for different steps of data curation while we furnish compilations of tools collected from a survey applied among members of the Society of Imaging Informatics (SIIM) for each of these stages. This collection has the potential to enhance the decision-making process for researchers as they select the most appropriate tool for their specific tasks.

Pseudo-spectral angle mapping for automated pixel-level analysis of highly multiplexed tissue image data.

The rapid development of highly multiplexed microscopy systems has enabled the study of cells embedded within their native tissue, which is providing exciting insights into the spatial features of human disease [1]. However, computational methods for analyzing these high-content images are still emerging, and there is a need for more robust and generalizable tools for evaluating the cellular constituents and underlying stroma captured by high-plex imaging [2]. To address this need, we have adapted spectral angle mapping - an algorithm used widely in hyperspectral image analysis - to compress the channel dimension of high-plex immunofluorescence images. As many high-plex immunofluorescence imaging experiments probe unique sets of protein markers, existing cell and pixel classification models do not typically generalize well. Pseudospectral angle mapping (pSAM) uses reference pseudospectra - or pixel vectors - to assign each pixel in an image a similarity score to several cell class reference vectors, which are defined by each unique staining panel. Here, we demonstrate that the class maps provided by pSAM can directly provide insight into the prevalence of each class defined by reference pseudospectra. In a dataset of high-plex images of colon biopsies from patients with gut autoimmune conditions, sixteen pSAM class representation maps were combined with instance segmentation of cells to provide cell class predictions. Finally, pSAM detected a diverse set of structure and immune cells when applied to a novel dataset of kidney biopsies imaged with a 43-marker panel. In summary, pSAM provides a powerful and readily generalizable method for evaluating high-plex immunofluorescence image data.

Role of sureness in evaluating AI/CADx: Lesion-based repeatability of machine learning classification performance on breast MRI.

BACKGROUND: Artificial intelligence/computer-aided diagnosis (AI/CADx) and its use of radiomics have shown potential in diagnosis and prognosis of breast cancer. Performance metrics such as the area under the receiver operating characteristic (ROC) curve (AUC) are frequently used as figures of merit for the evaluation of CADx. Methods for evaluating lesion-based measures of performance may enhance the assessment of AI/CADx pipelines, particularly in the situation of comparing performances by classifier. PURPOSE: The purpose of this study was to investigate the use case of two standard classifiers to (1) compare overall classification performance of the classifiers in the task of distinguishing between benign and malignant breast lesions using radiomic features extracted from dynamic contrast-enhanced magnetic resonance (DCE-MR) images, (2) define a new repeatability metric (termed sureness), and (3) use sureness to examine if one classifier provides an advantage in AI diagnostic performance by lesion when using radiomic features. METHODS: Images of 1052 breast lesions (201 benign, 851 cancers) had been retrospectively collected under HIPAA/IRB compliance. The lesions had been segmented automatically using a fuzzy c-means method and thirty-two radiomic features had been extracted. Classification was investigated for the task of malignant lesions (81% of the dataset) versus benign lesions (19%). Two classifiers (linear discriminant analysis, LDA and support vector machines, SVM) were trained and tested within 0.632 bootstrap analyses (2000 iterations). Whole-set classification performance was evaluated at two levels: (1) the 0.632+ bias-corrected area under the ROC curve (AUC) and (2) performance metric curves which give variability in operating sensitivity and specificity at a target operating point (95% target sensitivity). Sureness was defined as 1-95% confidence interval of the classifier output for each lesion for each classifier. Lesion-based repeatability was evaluated at two levels: (1) repeatability profiles, which represent the distribution of sureness across the decision threshold and (2) sureness of each lesion. The latter was used to identify lesions with better sureness with one classifier over another while maintaining lesion-based performance across the bootstrap iterations. RESULTS: In classification performance assessment, the median and 95% CI of difference in AUC between the two classifiers did not show evidence of difference (ΔAUC = -0.003 [-0.031, 0.018]). Both classifiers achieved the target sensitivity. Sureness was more consistent across the classifier output range for the SVM classifier than the LDA classifier. The SVM resulted in a net gain of 33 benign lesions and 307 cancers with higher sureness and maintained lesion-based performance. However, with the LDA there was a notable percentage of benign lesions (42%) with better sureness but lower lesion-based performance. CONCLUSIONS: When there is no evidence for difference in performance between classifiers using AUC or other performance summary measures, a lesion-based sureness metric may provide additional insight into AI pipeline design. These findings present and emphasize the utility of lesion-based repeatability via sureness in AI/CADx as a complementary enhancement to other evaluation measures.

Sequestration of imaging studies in MIDRC: stratified sampling to balance demographic characteristics of patients in a multi-institutional data commons.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) is a multi-institutional effort to accelerate medical imaging machine intelligence research and create a publicly available image repository/commons as well as a sequestered commons for performance evaluation and benchmarking of algorithms. After de-identification, approximately 80% of the medical images and associated metadata become part of the open commons and 20% are sequestered from the open commons. To ensure that both commons are representative of the population available, we introduced a stratified sampling method to balance the demographic characteristics across the two datasets. Approach: Our method uses multi-dimensional stratified sampling where several demographic variables of interest are sequentially used to separate the data into individual strata, each representing a unique combination of variables. Within each resulting stratum, patients are assigned to the open or sequestered commons. This algorithm was used on an example dataset containing 5000 patients using the variables of race, age, sex at birth, ethnicity, COVID-19 status, and image modality and compared resulting demographic distributions to naïve random sampling of the dataset over 2000 independent trials. Results: Resulting prevalence of each demographic variable matched the prevalence from the input dataset within one standard deviation. Mann-Whitney U test results supported the hypothesis that sequestration by stratified sampling provided more balanced subsets than naïve randomization, except for demographic subcategories with very low prevalence. Conclusions: The developed multi-dimensional stratified sampling algorithm can partition a large dataset while maintaining balance across several variables, superior to the balance achieved from naïve randomization.

Pilot study of machine learning in the task of distinguishing high and low-grade pediatric hydronephrosis on ultrasound.

PURPOSE: Hydronephrosis is a common pediatric urological condition, characterized by dilation of the renal collecting system. Accurate identification of the severity of hydronephrosis is crucial in clinical management, as high-grade hydronephrosis can cause significant damage to the kidney. In this pilot study, we demonstrate the feasibility of machine learning in differentiating between high and low-grade hydronephrosis in pediatric patients. MATERIALS AND METHODS: We retrospectively reviewed 592 images from 90 unique patients ages 0-8 years diagnosed with hydronephrosis at the University of Chicago's Pediatric Urology Clinic. The study included 74 high-grade hydronephrosis (145 images) and 227 low-grade hydronephrosis (447 images). Patients were excluded if they had less than 2 studies prior to surgical intervention or had structural abnormalities. We developed a radiomic-based artificial intelligence algorithm incorporating computerized texture analysis and machine learning (support-vector machine) to yield a predictor of hydronephrosis grade. RESULTS: Receiver operating characteristic analysis of the classifier output yielded an area under the curve value of 0.86 (95% CI 0.81-0.92) in the task of distinguishing between low and high-grade hydronephrosis using a five-fold cross-validation by kidney. In addition, a Mann-Kendall trend test between computer output and clinical hydronephrosis grade yielded a statistically significant upward trend (p<0.001). CONCLUSIONS: Our findings demonstrate the potential of machine learning in the differentiation between low and high-grade hydronephrosis. Further studies are warranted to validate our findings and their generalizability for use in clinical practice as a means to predict clinical outcomes and the resolution of hydronephrosis.

Predicting intensive care need for COVID-19 patients using deep learning on chest radiography.

Purpose: Image-based prediction of coronavirus disease 2019 (COVID-19) severity and resource needs can be an important means to address the COVID-19 pandemic. In this study, we propose an artificial intelligence/machine learning (AI/ML) COVID-19 prognosis method to predict patients' needs for intensive care by analyzing chest X-ray radiography (CXR) images using deep learning. Approach: The dataset consisted of 8357 CXR exams from 5046 COVID-19-positive patients as confirmed by reverse transcription polymerase chain reaction (RT-PCR) tests for the SARS-CoV-2 virus with a training/validation/test split of 64%/16%/20% on a by patient level. Our model involved a DenseNet121 network with a sequential transfer learning technique employed to train on a sequence of gradually more specific and complex tasks: (1) fine-tuning a model pretrained on ImageNet using a previously established CXR dataset with a broad spectrum of pathologies; (2) refining on another established dataset to detect pneumonia; and (3) fine-tuning using our in-house training/validation datasets to predict patients' needs for intensive care within 24, 48, 72, and 96 h following the CXR exams. The classification performances were evaluated on our independent test set (CXR exams of 1048 patients) using the area under the receiver operating characteristic curve (AUC) as the figure of merit in the task of distinguishing between those COVID-19-positive patients who required intensive care following the imaging exam and those who did not. Results: Our proposed AI/ML model achieved an AUC (95% confidence interval) of 0.78 (0.74, 0.81) when predicting the need for intensive care 24 h in advance, and at least 0.76 (0.73, 0.80) for 48 h or more in advance using predictions based on the AI prognostic marker derived from CXR images. Conclusions: This AI/ML prediction model for patients' needs for intensive care has the potential to support both clinical decision-making and resource management.

Radiomic and deep learning characterization of breast parenchyma on full field digital mammograms and specimen radiographs: a pilot study of a potential cancer field effect.

Purpose: In women with biopsy-proven breast cancer, histologically normal areas of the parenchyma have shown molecular similarity to the tumor, supporting a potential cancer field effect. The purpose of this work was to investigate relationships of human-engineered radiomic and deep learning features between regions across the breast in mammographic parenchymal patterns and specimen radiographs. Approach: This study included mammograms from 74 patients with at least 1 identified malignant tumor, of whom 32 also possessed intraoperative radiographs of mastectomy specimens. Mammograms were acquired with a Hologic system and specimen radiographs were acquired with a Fujifilm imaging system. All images were retrospectively collected under an Institutional Review Board-approved protocol. Regions of interest (ROI) of 128×128 pixels were selected from three regions: within the identified tumor, near to the tumor, and far from the tumor. Radiographic texture analysis was used to extract 45 radiomic features and transfer learning was used to extract 20 deep learning features in each region. Kendall's Tau-b and Pearson correlation tests were performed to assess relationships between features in each region. Results: Statistically significant correlations in select subgroups of features with tumor, near to the tumor, and far from the tumor ROI regions were identified in both mammograms and specimen radiographs. Intensity-based features were found to show significant correlations with ROI regions across both modalities. Conclusions: Results support our hypothesis of a potential cancer field effect, accessible radiographically, across tumor and non-tumor regions, thus indicating the potential for computerized analysis of mammographic parenchymal patterns to predict breast cancer risk.

Machine learning with multimodal data for COVID-19.

In response to the unprecedented global healthcare crisis of the COVID-19 pandemic, the scientific community has joined forces to tackle the challenges and prepare for future pandemics. Multiple modalities of data have been investigated to understand the nature of COVID-19. In this paper, MIDRC investigators present an overview of the state-of-the-art development of multimodal machine learning for COVID-19 and model assessment considerations for future studies. We begin with a discussion of the lessons learned from radiogenomic studies for cancer diagnosis. We then summarize the multi-modality COVID-19 data investigated in the literature including symptoms and other clinical data, laboratory tests, imaging, pathology, physiology, and other omics data. Publicly available multimodal COVID-19 data provided by MIDRC and other sources are summarized. After an overview of machine learning developments using multimodal data for COVID-19, we present our perspectives on the future development of multimodal machine learning models for COVID-19.

Longitudinal assessment of demographic representativeness in the Medical Imaging and Data Resource Center open data commons.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) open data commons was launched to accelerate the development of artificial intelligence (AI) algorithms to help address the COVID-19 pandemic. The purpose of this study was to quantify longitudinal representativeness of the demographic characteristics of the primary MIDRC dataset compared to the United States general population (US Census) and COVID-19 positive case counts from the Centers for Disease Control and Prevention (CDC). Approach: The Jensen-Shannon distance (JSD), a measure of similarity of two distributions, was used to longitudinally measure the representativeness of the distribution of (1) all unique patients in the MIDRC data to the 2020 US Census and (2) all unique COVID-19 positive patients in the MIDRC data to the case counts reported by the CDC. The distributions were evaluated in the demographic categories of age at index, sex, race, ethnicity, and the combination of race and ethnicity. Results: Representativeness of the MIDRC data by ethnicity and the combination of race and ethnicity was impacted by the percentage of CDC case counts for which this was not reported. The distributions by sex and race have retained their level of representativeness over time. Conclusion: The representativeness of the open medical imaging datasets in the curated public data commons at MIDRC has evolved over time as the number of contributing institutions and overall number of subjects have grown. The use of metrics, such as the JSD support measurement of representativeness, is one step needed for fair and generalizable AI algorithm development.

Temporal Machine Learning Analysis of Prior Mammograms for Breast Cancer Risk Prediction.

The identification of women at risk for sporadic breast cancer remains a clinical challenge. We hypothesize that the temporal analysis of annual screening mammograms, using a long short-term memory (LSTM) network, could accurately identify women at risk of future breast cancer. Women with an imaging abnormality, which had been biopsy-confirmed to be cancer or benign, who also had antecedent imaging available were included in this case-control study. Sequences of antecedent mammograms were retrospectively collected under HIPAA-approved guidelines. Radiomic and deep-learning-based features were extracted on regions of interest placed posterior to the nipple in antecedent images. These features were input to LSTM recurrent networks to classify whether the future lesion would be malignant or benign. Classification performance was assessed using all available antecedent time-points and using a single antecedent time-point in the task of lesion classification. Classifiers incorporating multiple time-points with LSTM, based either on deep-learning-extracted features or on radiomic features, tended to perform statistically better than chance, whereas those using only a single time-point failed to show improved performance compared to chance, as judged by area under the receiver operating characteristic curves (AUC: 0.63 ± 0.05, 0.65 ± 0.05, 0.52 ± 0.06 and 0.54 ± 0.06, respectively). Lastly, similar classification performance was observed when using features extracted from the affected versus the contralateral breast in predicting future unilateral malignancy (AUC: 0.63 ± 0.05 vs. 0.59 ± 0.06 for deep-learning-extracted features; 0.65 ± 0.05 vs. 0.62 ± 0.06 for radiomic features). The results of this study suggest that the incorporation of temporal information into radiomic analyses may improve the overall classification performance through LSTM, as demonstrated by the improved discrimination of future lesions as malignant or benign. Further, our data suggest that a potential field effect, changes in the breast extending beyond the lesion itself, is present in both the affected and contralateral breasts in antecedent imaging, and, thus, the evaluation of either breast might inform on the future risk of breast cancer.

Patient-specific fetal radiation dosimetry for pregnant patients undergoing abdominal and pelvic CT imaging.

BACKGROUND: Accurate estimation of fetal radiation dose is crucial for risk-benefit analysis of radiological imaging, while the radiation dosimetry studies based on individual pregnant patient are highly desired. PURPOSE: To use Monte Carlo calculations for estimation of fetal radiation dose from abdominal and pelvic computed tomography (CT) examinations for a population of patients with a range of variations in patients' anatomy, abdominal circumference, gestational age (GA), fetal depth (FD), and fetal development. METHODS: Forty-four patient-specific pregnant female models were constructed based on CT imaging data of pregnant patients, with gestational ages ranging from 8 to 35 weeks. The simulation of abdominal and pelvic helical CT examinations was performed on three validated commercial scanner systems to calculate organ-level fetal radiation dose. RESULTS: The absorbed radiation dose to the fetus ranged between 0.97 and 2.24 mGy, with an average of 1.63 ± 0.33 mGy. The CTDIvol -normalized fetal dose ranged between 0.56 and 1.30, with an average of 0.94 ± 0.25. The normalized fetal organ dose showed significant correlations with gestational age, maternal abdominal circumference (MAC), and fetal depth. The use of ATCM technique increased the fetal radiation dose in some patients. CONCLUSION: A technique enabling the calculation of organ-level radiation dose to the fetus was developed from models of actual anatomy representing a range of gestational age, maternal size, and fetal position. The developed maternal and fetal models provide a basis for reliable and accurate radiation dose estimation to fetal organs.

Evaluation of emphysema on thoracic low-dose CTs through attention-based multiple instance deep learning.

In addition to lung cancer, other thoracic abnormalities, such as emphysema, can be visualized within low-dose CT scans that were initially obtained in cancer screening programs, and thus, opportunistic evaluation of these diseases may be highly valuable. However, manual assessment for each scan is tedious and often subjective, thus we have developed an automatic, rapid computer-aided diagnosis system for emphysema using attention-based multiple instance deep learning and 865 LDCTs. In the task of determining if a CT scan presented with emphysema or not, our novel Transfer AMIL approach yielded an area under the ROC curve of 0.94 ± 0.04, which was a statistically significant improvement compared to other methods evaluated in our study following the Delong Test with correction for multiple comparisons. Further, from our novel attention weight curves, we found that the upper lung demonstrated a stronger influence in all scan classes, indicating that the model prioritized upper lobe information. Overall, our novel Transfer AMIL method yielded high performance and provided interpretable information by identifying slices that were most influential to the classification decision, thus demonstrating strong potential for clinical implementation.

Differences in Molecular Subtype Reference Standards Impact AI-based Breast Cancer Classification with Dynamic Contrast-enhanced MRI.

Background Breast cancer tumors can be identified as different luminal molecular subtypes depending on either immunohistochemical (IHC) staining or St Gallen criteria that includes Ki-67. Purpose To characterize molecular subtypes and understand the impact of disagreement among IHC and St Gallen molecular subtype reference standards on artificial intelligence classification of luminal A and luminal B tumors with use of radiomic features extracted from dynamic contrast-enhanced (DCE) MRI scans. Materials and Methods In this retrospective study, 28 radiomic features previously extracted from DCE-MRI scans of breast tumors imaged between February 2015 and October 2017 were examined in the following groups: (a) tumors classified as luminal A by both reference standards ("agreement"), (b) tumors classified as luminal A by IHC and luminal B by St Gallen ("disagreement"), and (c) tumors classified as luminal B by both ("agreement"). Luminal A or luminal B tumor classification with use of radiomic features was conducted with use of three sets: (a) IHC molecular subtyping, (b) St Gallen molecular subtyping, and (c) agreement tumors. The Kruskal-Wallis test was followed by the Mann-Whitney U test to determine pair-wise differences of radiomic features among agreement and disagreement tumors. Fivefold cross-validation with use of stepwise feature selection and linear discriminant analysis classified tumors in each set, with performance measured with use of area under the receiver operating characteristic curve (AUC). Results A total of 877 breast cancer tumors from 872 women (mean age, 48 years [range, 19-75 years]) were analyzed. Six features (sphericity, irregularity, surface area to volume ratio, variance of radial gradient histogram, sum average, volume of most enhancing voxels) were different (P ≤ .001) among agreement and disagreement tumors. AUC (median, 0.74 [95% CI: 0.68, 0.80]) was higher than when using tumors subtyped by either reference standard (IHC, 0.66 [0.60, 0.71], P = .003; St Gallen, 0.62 [0.58, 0.67], P = .001). Conclusion Differences in reference standards can hinder artificial intelligence classification performance of luminal molecular subtypes with dynamic contrast-enhanced MRI. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Bae in this issue.

Criteria for the translation of radiomics into clinically useful tests.

Computer-extracted tumour characteristics have been incorporated into medical imaging computer-aided diagnosis (CAD) algorithms for decades. With the advent of radiomics, an extension of CAD involving high-throughput computer-extracted quantitative characterization of healthy or pathological structures and processes as captured by medical imaging, interest in such computer-extracted measurements has increased substantially. However, despite the thousands of radiomic studies, the number of settings in which radiomics has been successfully translated into a clinically useful tool or has obtained FDA clearance is comparatively small. This relative dearth might be attributable to factors such as the varying imaging and radiomic feature extraction protocols used from study to study, the numerous potential pitfalls in the analysis of radiomic data, and the lack of studies showing that acting upon a radiomic-based tool leads to a favourable benefit-risk balance for the patient. Several guidelines on specific aspects of radiomic data acquisition and analysis are already available, although a similar roadmap for the overall process of translating radiomics into tools that can be used in clinical care is needed. Herein, we provide 16 criteria for the effective execution of this process in the hopes that they will guide the development of more clinically useful radiomic tests in the future.

Construction of a digital fetus library for radiation dosimetry.

PURPOSE: Accurate estimations of fetal absorbed dose and radiation risks are crucial for radiation protection and important for radiological imaging research owing to the high radiosensitivity of the fetus. Computational anthropomorphic models have been widely used in patient-specific radiation dosimetry calculations. In this work, we aim to build the first digital fetal library for more reliable and accurate radiation dosimetry studies. ACQUISITION AND VALIDATION METHODS: Computed tomography (CT) images of abdominal and pelvic regions of 46 pregnant females were segmented by experienced medical physicists. The segmented tissues/organs include the body contour, skeleton, uterus, liver, kidney, intestine, stomach, lung, bladder, gall bladder, spleen, and pancreas for maternal body, and placenta, amniotic fluid, fetal body, fetal brain, and fetal skeleton. Nonuniform rational B-spline (NURBS) surfaces of each identified region was constructed manually using 3D modeling software. The Hounsfield unit values of each identified organs were gathered from CT images of pregnant patients and converted to tissue density. Organ volumes were further adjusted according to reference measurements for the developing fetus recommended by the World Health Organization (WHO) and International Commission on Radiological Protection. A series of anatomical parameters, including femur length, humerus length, biparietal diameter, abdominal circumference (FAC), and head circumference, were measured and compared with WHO recommendations. DATA FORMAT AND USAGE NOTES: The first fetal patient-specific model library was developed with the anatomical characteristics of each model derived from the corresponding patient whose gestational age varies between 8 and 35 weeks. Voxelized models are represented in the form of MCNP matrix input files representing the three-dimensional model of the fetus. The size distributions of each model are also provided in text files. All data are stored on Zenodo and are publicly accessible on the following link: https://zenodo.org/record/6471884. POTENTIAL APPLICATIONS: The constructed fetal models and maternal anatomical characteristics are consistent with the corresponding patients. The resulting computational fetus could be used in radiation dosimetry studies to improve the reliability of fetal dosimetry and radiation risks assessment. The advantages of NURBS surfaces in terms of adapting fetal postures and positions enable us to adequately assess their impact on radiation dosimetry calculations.

Assessment of a deep learning model for COVID-19 classification on chest radiographs: a comparison across image acquisition techniques and clinical factors.

Purpose: The purpose is to assess the performance of a pre-trained deep learning model in the task of classifying between coronavirus disease (COVID)-positive and COVID-negative patients from chest radiographs (CXRs) while considering various image acquisition parameters, clinical factors, and patient demographics. Methods: Standard and soft-tissue CXRs of 9860 patients comprised the "original dataset," consisting of training and test sets and were used to train a DenseNet-121 architecture model to classify COVID-19 using three classification algorithms: standard, soft tissue, and a combination of both types of images via feature fusion. A larger more-current test set of 5893 patients (the "current test set") was used to assess the performance of the pretrained model. The current test set contained a larger span of dates, incorporated different variants of the virus and included different immunization statuses. Model performance between the original and current test sets was evaluated using area under the receiver operating characteristic curve (ROC AUC) [95% CI]. Results: The model achieved AUC values of 0.67 [0.65, 0.70] for cropped standard images, 0.65 [0.63, 0.67] for cropped soft-tissue images, and 0.67 [0.65, 0.69] for both types of cropped images. These were all significantly lower than the performance of the model on the original test set. Investigations regarding matching the acquisition dates between the test sets (i.e., controlling for virus variants), immunization status, disease severity, and age and sex distributions did not fully explain the discrepancy in performance. Conclusions: Several relevant factors were considered to determine whether differences existed in the test sets, including time period of image acquisition, vaccination status, and disease severity. The lower performance on the current test set may have occurred due to model overfitting and a lack of generalizability.

Special Issue Editorial: The SPIE Medical Imaging Symposium Celebrates 50 Years.

The article introduces the JMI Special Issue Celebrating 50 Years of SPIE Medical Imaging.

AAPM Task Group 298: Recommendations on certificate program/alternative pathway candidate education and training.

Entry into the field of clinical medical physics is most commonly accomplished through the completion of a Commission on Accreditation of Medical Physics Educational Programs (CAMPEP)-accredited graduate and residency program. To allow a mechanism to bring valuable expertise from other disciplines into clinical practice in medical physics, an "alternative pathway" approach was also established. To ensure those trainees who have completed a doctoral degree in physics or a related discipline have the appropriate background and didactic training in medical physics, certificate programs and a CAMPEP-accreditation process for these programs were initiated. However, medical physics-specific didactic, research, and clinical exposure of those entering medical physics residencies from these certificate programs is often comparatively modest when evaluated against individuals holding Master's and/or Doctoral degrees in CAMPEP-accredited graduate programs. In 2016, the AAPM approved the formation of Task Group (TG) 298, "Alternative Pathway Candidate Education and Training." The TG was charged with reviewing previous published recommendations for alternative pathway candidates and developing recommendations on the appropriate education and training of these candidates. This manuscript is a summary of the AAPM TG 298 report.

A machine-learning algorithm for distinguishing malignant from benign indeterminate thyroid nodules using ultrasound radiomic features.

Background: Ultrasound (US)-guided fine needle aspiration (FNA) cytology is the gold standard for the evaluation of thyroid nodules. However, up to 30% of FNA results are indeterminate, requiring further testing. In this study, we present a machine-learning analysis of indeterminate thyroid nodules on ultrasound with the aim to improve cancer diagnosis. Methods: Ultrasound images were collected from two institutions and labeled according to their FNA (F) and surgical pathology (S) diagnoses [malignant (M), benign (B), and indeterminate (I)]. Subgroup breakdown (FS) included: 90 BB, 83 IB, 70 MM, and 59 IM thyroid nodules. Margins of thyroid nodules were manually annotated, and computerized radiomic texture analysis was conducted within tumor contours. Initial investigation was conducted using five-fold cross-validation paradigm with a two-class Bayesian artificial neural networks classifier, including stepwise feature selection. Testing was conducted on an independent set and compared with a commercial molecular testing platform. Performance was evaluated using receiver operating characteristic analysis in the task of distinguishing between malignant and benign nodules. Results: About 1052 ultrasound images from 302 thyroid nodules were used for radiomic feature extraction and analysis. On the training/validation set comprising 263 nodules, five-fold cross-validation yielded area under curves (AUCs) of 0.75 [Standard Error (SE) = 0.04; P < 0.001 ] and 0.67 (SE = 0.05; P = 0.0012 ) for the classification tasks of MM versus BB, and IM versus IB, respectively. On an independent test set of 19 IM/IB cases, the algorithm for distinguishing indeterminate nodules yielded an AUC value of 0.88 (SE = 0.09; P < 0.001 ), which was higher than the AUC of a commercially available molecular testing platform (AUC = 0.81, SE = 0.11; P < 0.005 ). Conclusion: Machine learning of computer-extracted texture features on gray-scale ultrasound images showed promising results classifying indeterminate thyroid nodules according to their surgical pathology.